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21 AUG 1981 


MEMORANDUM FOR THE RECORD 
SUBJECT: Utilities for ODP Computer Centers 


REFERENCE: Memo for D/L from D/ODP dtd 3 Aug 81, 
Same Subject 


1. Problem: 


By referent, the Director of ODP claims that electrical 
power outages are degrading his computer operations by one third 
of the time, citing 20 failures in two years. He requests 
immediate scheduling of all maintenance and repair outside of 
prime time working hours, and reduction of power failures to no 
more than one failure per year. 


ae Background: 


ODP has long been interested in reliable power. It 
has also been in a dynamic mode requiring major replacement of 
hardware in GC03 and GC47, expansion into 1D16, expansion of 
1D16, second expansion of 1D16, not to mention projects SAFE, 
CAMS, APARS, etc. 


Each hardware upgrade has been accompanied by a major 
impact of facilities such as power and air conditioning. Wire 
closets have had to be moved, transformer vaults enlarged, and 
capacity and reliability enhancing equipment installed all the 
way from the powerhouse to vaults A, B, and C. Initially, 
power outages were addressed by installation of generators. 

This was followed by installation of motor generator sets to 
provide both isolation and frequency conversion from 60 to A415 

Hz. Motor generators were succeeded by solid state uninterruptible 
power systems with increased capacity required at each step. 

During this constant evolution process, it has been necessary to 
work on energized circuits, work with redundant feeders tempor- 
arily out of service, and through it all to test both incrementally 
and by system that which has been built to enhance reliability or 
keep up with load growth. In order to protect against prolonged 
power outages, generators have been installed sufficient to run 

all critical loads including the powerplant and its air con- 
ditioning equipment. On top of the above Agency initiated act- 
ivities, the Virginia Electric and Power Company (VEPCO) has 
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been obliged to upgrade its facilities feeding the McLean/CIA 
area. For the past two years, major construction by VEPCO has 
had our main power feeders operating at 50% reliability while 
the entire substation and miles of transmission line has been 
completely dismantled and is only at this writing being 
replaced. 


To respond affirmatively to the referent, it would 
be necessary to acknowledge that the majority of interruptions 
come from outside areas under Agency control. It would be 
theoretically necessary to perfect the very equipment described 
above. Of course, if that were possible, there would be no 
problem. In fact, however, the existence of the equipment to 
enhance reliability places electro/mechanical devices in an 
otherwise hard-wired circuit. These devices have a variety of 
faétory tolerances and in combination or individually have a 
variety of mean time between failure characteristics well short 
of infinity. When, in spite of all, a failure does occur ,and 
even if a successful switch to bypass is made, not repairing 
or correcting during "prime time" does not of itself guarantee 
the absence of a subsequent failure. Conversely, it does 
guarantee that if a VEPCO failure occurs before repairs are 
accomplished that ODP and other customers will be directly 
affected. Also, the efficiency of accomplishing a repair is 
not expected to be increased given the increased problem of 
obtaining access, escorts, parts, or even factory technicians 
who are not dedicated to just CIA plus the supporting electri- 
cians or operating engineers who must manipulate switches or 
valves to produce a necessary effect at the desired time. 


5.3 Conclusion: 


The Director of ODP's indication of concern and will- 
ingness to assist in the form of resources should be utilized 
to further strengthen our efforts to approach absolute utility 
reliability. Given the occasional pervisity of circumstance 
and equipment involved the D/L should take extreme caution in 
making guarantees which are beyond a reasonable ability to 
deliver. Moreover, the D/L should also be careful to retain 
the right to do what is necessary in the event of a utility 
problem while extending the courtesy of consultation and 
coordination as the norm. 


Deputy Chief, Real Estate and 
Construction Division, OL 
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MEMORANDUM FOR: Director of Logistics 
15 AUG i333 


epg t a : = . ! ‘ { 
FROM: Bruce T, Johnson 
Director of Data Processing 


SUBJECT: Utilities for Ob? computer Centers 
1. Action Requested: 


We request that the Office of Logistics take the necessary 
steps to lower the rate of utility failures to no more than 
one per year for each of the ODP computer centers. As an 
immediate measure scheduled maintenance and scheduled repair 
should be conducted on ODP utilities outside of prime time 
working hours, 


1 

2. Background: | 

ODP has made a concentrated effort during the last 24 
months to increase the time that the ODP facilities are available 
to our customers. Needless to say, the quality of the utilitias 
that drive the computer centers greatly affects our ability to 
keep our systems operating. Unfortunately, the number of 
failures during the last two years suggest that there is a serious 
problem with the quality of this service. . 


During this time frame, ODP has experienced 20 failures due 
to utility outages. Ten of these failuras have occurred since 
31 March 1981. With each of these failures we have lost an 
average of four hours of processing. What is even more important, 
however, is that we have experienced a significant increase in 
the rate of equipment failure. 4 recent analysis has shown that 
for two weeks following an unscheduled power outage. the rate 
of equipment failure is two to three timos the normal rate. 
When you consider that we had 20 failures in two years, the Opp 
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computer centers have been operating in degraded mode at best for 
over one-third of the time because of power failures (20 x 2 weeks 
divided by 108 weeks}. We cannot expect our users te continue to 
tolerate such a situation. We must have more dependable power, 

and I earnestly request the Office of Logistics to design utility 
systems such that we experience no more than one failure per year. 


As an immediate measuré, I must ask that all. OReaui oa 
maintenance on ODP utilities be accomplished outside af the 
hours of 0700-2000, Monday-Friday to avoid the high risk of 
inadvertently disrupting essential service when naaRES anance 
is performed during the prime shift. 


Undoubtedly there will be resource issues raised by this 


request. We will be happy to work with you to resolve these 
issues and to obtain the necessary funds. 


STAT 
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